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to  Identify  and  correct  deficiencies  in  individual  repair  skill. 
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TECHNICAL  SUMMARY 


PROBLEM 

The  Army  emphasizes  on-the-job  training  (OJT)  as  a  way  to  develop  the 
proficiency  of  battalion  and  company  maintenance  personnel.  However,  OJT 
programs  seldom  include  ways  to  measure  mechanic  proficiency.  Measuring  OJT  is 
not  done  because  accurate  measures  are  difficult  to  administer.  Easy-to- 
administer  n  asures  are  suspect  in  terms  of  validity  and  reliability.  Therefore, 
without  satisfactory  proficiency  measures,  skill  deficiencies  cannot  be  identified 
for  correction. 

One  component  of  the  Maintenance  Performance  System  (Organizational), 
MPS(O)),  proposes  to  measure  the  proficiency  of  individual  mechanics  by 
monitoring  their  job  exposure.  Job  exposure  measurement  assumes  that  a 
mechanic’s  proficiency  is  increased  through  the  practice  gained  in  repeated 
exposures  to  the  same  job.  However,  the  relationship  between  proficiency  and  job 
exposure  has  not  been  tested  empirically.  The  research  reported  here  was 
conducted  to  assess  this  relationship. 

APPROACH 

The  relationship  between  job  exposure  and  maintenance  proficiency  (i.e., 
skill  level)  was  examined  in  a  controlled  experimental  study.  A  total  of  70  tank 
auto  me  dve  mechanics  (mast  with  MOS  63N)  were  individually  tested  on  two  M60A1 
tank  r  'pair  tasks — starter  installation  and  generator  installation.  Testing  was 
conduc  ted  by  an  Army  battalion  maintenance  technician  with  20  years  of  experi¬ 
ence  .rho  had  retired  recently.  The  mechanics  tested  had  performed  the  tasks 
either  0,  l,  2,  3,  4,  5,  6,  7,  8,  or  9  or  more  times  prior  to  testing.  Maintenance 
profie  ency  measures  consisted  of:  use  of  the  technical  manual,  adherence  to  the 
prescribed  repair  sequence,  tool  selection,  tool  use,  checks. of  repair  quality,  and 
repair  time. 

RESULTS 

The  study  produced  the  following  results: 

9  Performance  frequency  was  strongly  related  to  maintenance  proficiency, 
accounting  for  one-half  the  total  variation  in  the  test  data. 

o  iimilar  functional  relationships  were  found  between  performance  fre¬ 
quency  and  six  different  measures  of  maintenance  work,  suggesting  that 
performance  'requency  was  strongly  related  to  afi  components  oi  mainte¬ 
nance  proficiency.  y. 

: 

m  The  function  relating  maintenance  proficiency  to  performance  frequency 
was  similar  for  two  different  maintenance  tasks,  suggesting  that  the 
obser  ■•‘d  function  is  reliable  <  id  valid.  . 
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•  overall,  maintenance  proficiency  increased  with  additional  task  per- 
formano^s,  peaked  between  five  and  six  performances,  and  declined 
sightly  beyond  six  performances  as  shown  in  the  figure  helow. 


NUMBER  OF  PREVIOUS  TASK  PERFORMANCES 


Relationship  of  maintenance  proficiency  to  performance  frequency. 


CONC  jUSIOKS 


•  Performance  frequency  is  an  accurate  and  useful  indicator  of  mainte¬ 
nance  proficiency.  _  It  is  strongly  related  to  six  different  dimensions  of 
maintenance  proficiency— quality  checks,  adherence  to  procedures,  repair 
time,  information  use,  tool  selection,  and  tool  use. 

•  T,h0  J?b.  exP°^re  index,  derived  from  records  of  performance  frequency 
!h°ul.d,  retained  in  the  MPS(O)  for  assessing  skill  growth,  providing 
individual  skill  profiles,  and  identifying  skill  development  needs. 

•  Comments  made  by  subjects  about  the  testing  suggested  that  elements  of 
the  controlled  testing  environment  could  be  effectively  transferred  as 
components  of  an  OJ'T program. 

e  Farther  research  is  needed  to  determine  the  reasons  for  the  decline  in 
pr  ficien  y  displayed  consistently  by  the  mechanics  with  greatest  job 
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INTRODUCTION 


This  report  concerns  a  unique  way  of  measuring  job  skill.  Whereas  formal 
hands-on  tests  of  a  mechanic's  repair  performance  are  widely  acknowledged  to  be 
accurate  and  valid  reflections  of  skill,  they  consume  too  many  resources  to  be 
efficiently  applied  on  a  large  scale.  As  part  of  its  long-term  effort  to  improve 
company-level  maintenance,  the  Maintenance  Performance  Systems  (Organiza¬ 
tional)  (MPS(O))  is  planning  to  use  a  simple,  easy- to- collect  substitute  for  the 
formal,  hands-on  skill  test — the  mechanic’s  number  of  previous  job  experiences  on 
a  task.  The  search  for  an  economical,  objective  skill  measure  was  prompted  by  the 
Army's  need  to  guide  on-the-job  training  (OJT)  programs  at  the  company  and 
battalion  level.  Monitoring  skill  growth  and  managing  OJT  are  two  of  the  primary 
goals  of  the  MPS(O). 

A  study  was  conducted  to  determine  if  job  exposure — defined  as  the  number 
of  times  a  task  has  been  performed — is,  in  fact,  an  appropriate  measure  of  skill. 
Although  job  exposure  (JE)  is  hypothesized  to  reflect  repair  skill,  is  it  a  valid 
measure?  If  it  is  not,  then  it  should  not  be  used  in  the  MPS(O).  The  present  study 
asked  whether  job  exposure  was  a  valid  index  of  skill  in  the  context  of  four 
separate,  but  related,  questions. 

•  How  closely  related  are  JE  and  repair  skill? 

•  What  is  the  quantitative  relationship  between  JE  and  repair  skill? 

•  Is  there  a  reliable  relationship  between  JE  and  repair  skill  that  can  be 
generalized  to  different  maintenance  tasks? 

•  Does  job  exposure  reflect  a  single  dimension,  or  does  it  reflect  most  of 
the  dimensions  of  repair  skill — quality,  repair  time,  tool  selection,  use  of 
tools,  use  of  information,  and  adherence  to  procedures? 

Although  it  seems  reasonable  to  assume  that  more  JEs  lead  to  better  repair 
skill,  the  two  are  not  necessarily  related.  Job  experience  may  be  accrued  too 
haphazardly,  individuals  may  vary  too  much  in  their  rate  of  skill  acquisition,  and 
poor  repair  habits  may  get  ingrained  too  quickly  and  cloud  the  real  relationship 
between  JE  and  skill.  Since  there  was  little  available  scientific  evidence  from 
which  to  draw  firm  conclusions,  the  work  reported  here  was  designed  to  fill  this 


gap.  Before  describing  the  study's  methodology  and  results,  the  role  of  JE  in  the 
MPS(O)  and  results  of  recently  completed  research  directly  relevant  to  this  issue 
are  discussed  below. 


BACKGROUND 

According  to  the  OJT  model  developed  for  the  MPS(O),*  the  company-level 
training  manager  implements  OJT  in  three  steps:  He  determines  the  maintenance 
proficiency  level  for  each  mechanic,  he  identifies  those  repair  tasks  that  each  man 
has  not  yet  mastered,  and  then  he  designs  an  individualized  training  program  to 
correct  those  skill  deficiencies.  The  key  step  in  the  OJT  process  is  determining 
each  mechanic's  current  skill  level,  i.e.,  how  well  can  a  mechanic  perform  each 
repair  task? 

Job  Exposure  and  the  MPSfO) 

In  the  current  version  of  the  MPS(O),  JE  is  employed  as  an  indicator  of 
repair  skill.  For  a  given  task,  a  mechanic  with  2  JEs  is  considered  to  be  more 
skilled  than  a  mechanic  with  1  JE,  but  less  skilled  than  a  mechanic  with  3  JEs. 
Every  time  a  mechanic  performs  a  task,  the  system  increases  his  JE  index  by  one. 
The  MPS(O)  provides  regular  listings  of  each  mechanic's  JE  count  on  every  repair 
task.  By  reviewing  the  list,  the  mechanic's  supervisor  can  determine  those  tasks  in 
need  of  OJT  or  JE.  Table  1  illustrates  the  link  between  JE  and  skill  development 
recommendations  for  a  few  63N  corrective  maintenance  tasks.  The  tasks  are  listed 
in  descending  order  of  their  need  for  OJT  or  JE. 

Previous  Research 

The  assumption  that  JE  can  be  used  to  substitute  for  a  direct  measure  of 

2 

repair  skill  has  been  challenged  in  a  recent  study  by  the  Army  Research  Institute. 


Simpson,  H.  K.  Maintenance  performance  systems  (organizational).  Supervisor's 
OJT  guide.  Santa  Barbara,  CA:  Anacapa  Sciences,  Inc.  (Technical  Report  465-21), 
June  1982. 

2 

Kern,  R.  P.,  &  Hayes,  J.  F.  Improving  maintenance  performance:  Application  of 
research  to  operational  problems.  Alexandria,  Virginia:  U.S.  Army  Research 
Institute  for  the  Behavioral  Sciences,  February  1982. 
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TABLE  1 


EXAMPLE  OF  CORRESPONDENCE  BETWEEN  JOB  EXPOSURE 
AND  NEED  FOR  OJT  FOR  A  63N  TANK  MECHANIC 


TYPICAL  REPAIR  TASKS 

JOB  EXPOSURE 

INTERPRETATION 

1. 

Replace  Starter 

0 

Needs  OJT  or  JE 

2. 

Adjust  Servo  Bands 

0 

Needs  OJT  or  JE 

3. 

Replace  Generator 

1 

Needs  OJT  or  JE 

4. 

Replace  Master  or  Slave  Cylinder 

1 

Needs  OJT  or  JE 

5. 

Replace  Shock  Absorber 

2 

May  need  supervision 

6. 

Repair  Wiring 

3 

May  need  supervision 

7. 

Bleed  Brake  Lines 

4 

May  need  supervision 

8. 

Replace  Parking  Brake  Cable 

6 

May  need  supervision 

9. 

Remove  Powerpack 

7 

Achieved  Mastery 

10. 

Ground  Hop  Powerpack 

7 

Achieved  Mastery 

11. 

Install  Powerpack 

7 

Achieved  Mastery 

12. 

Remove  Back  Deck 

7 

Achieved  Mastery 

Trained  technicians  went  into  company  motor  pools  to  observe  tank  mechanics 
perform  corrective  maintenance  tasks  under  operational  conditions.  Detailed 
records  were  kept  on  the  number,  and  type,  of  maintenance  errors  committed  on 
the  job.  Critical  errors  were  found  to  be  committed  frequently  in  most  mainte¬ 
nance  jobs,  and  experienced  mechanics  were  found  to  make  as  many  errors  as 
beginning  mechanics.  The  researchers  concluded  that  a  mechanic  does  not  become 
more  proficient  with  practice.  They  contended  that  mechanics  typically  received 
little  feedback  on  the  quality  of  their  work,  resulting  in  the  learning  and  retention 
of  bad  repair  habits.  As  an  assessment  of  repair  performance,  the  study  provided 
valuable  insights  into  the  problems  faced  by  mechanics  in  operational  conditions. 
But  because  repairs  were  observed  under  poorly  controlled  conditions,  and  the  level 
of  difficulty  of  the  repairs  varied,  it  is  not  possible  to  determine  the  relationship 
between  job  exposure  and  skill  levels  from  these  data. 
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ANTICIPATED  OUTCOMES  OF  THE  CURRENT  STUDY 

Four  possible  outcomes  for  the  quantitative  relationship  between  JE  and 
repair  skill  were  anticipated  prior  to  the  current  study.  Each  is  described  below: 


1.  JE  is  unrelated  to  repair  skill.  Figure  1A  shows  repair  skill  to  be  a  flat 
function  of  JE.  This  function  is  consistent  with  the  results  of  the  study 
discussed  above,  and  would  invalidate  the  use  of  JE  in  the  MPS(O). 
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JOB  EXPOSURE 


Figure  1A. 


2.  Repair  skill  first  increases,  then  levels  off.  The  relationship  portrayed  in 
Figure  IB  shows  that  repair  skill  reaches  a  plateau  after  a  finite  number 
of  JEs  and  does  not  increase. 


Figure  IB. 
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3.  Repair  skill  increases  indefinitely  with  JE.  Figure  1C  shows  the  case  in 
which  a  mechanic's  repair  skill  improves  continuously  with  more'JEs. 
This  relationship  implies  that  even  highly  experienced  mechanics  could 
benefit  from  further  JE. 


Figure  1C. 

4.  Repair  skill  first  increases,  then  decreases  slightly  as  a  function  of  JE. 
Figure  ID  depicts  this  relationship  between  JE  and  repair  skill.  A 
decline  in  skill  among  the  more  experienced  mechanics  would  be 
consistent  with  anecdotal  reports  that,  over  time,  these  men  become 
careless  and  cynical,  lose  motivation,  and  let  their  skills  atrophy. 


JOB  EXPOSURE 


Figure  ID. 


METHOD 


STUDY  DESIGN 

The  relationship  between  JE  and  repair  skill  was  examined  using  a  between- 
subjects  type  of  experimental  design.  Subjects  were  mechanics  who  routinely 
performed  corrective  maintenance  on  the  M60A1  tank  engine;  most  had  a  63N 
MOS.  To  increase  study  generalizability,  two  different  tank  repair  tasks  were 
employed — installing  a  starter,  and  installing  a  generator. 

Only  about  70  mechanics  were  available  for  testing,  which  restricted  the 
number  of  subjects  per  JE  category  and  the  range  of  JE  categories  that  could  be 
included.  A  study  design  was  therefore  adopted  that,  within  the  practical 
constraints  imposed  by  field  testing,  maximized  the  sample's  statistical  reliability 
over  the  most  critical  range  of  JEs.  A  minimum  of  five  subjects  was  drawn  from 
each  of  ten  categories  of  JE;  0,  1,  2,  3,  4,  5,  6,  7,  8,  and  9  or  more.  The  minimum 
sample  size  was  doubled  in  the  two  outside  JE  categories,  0,  and  9  or  more  (9+),  for 
the  following  statistical  reasons: 

•  To  compensate  for  the  limited  numbers  of  subjects  in  the  experimental 
design,  it  was  intended  that  the  data  for  subjects  in  adjacent  JE 
categories  be  combined  in  the  statistical  analysis.  Since  subjects  in  the 
two  outside  JE  categories  represented  a  different  statistical  population 
from  those  in  the  middle  JE  categories,  their  data  could  not  be  pooled 
with  those  from  another  category.  These  two  JE  categories  therefore 
required  more  subjects  to  stand  alone. 

•  There  are  more  mechanics  having  either  none,  or  many,  JEs  than  there 
are  having  moderate  numbers  of  JEs.  Increasing  the  sample  in  the  two 
outside  JE  categories  reflected  the  JE  of  the  actual  population  of 
mechanics. 

A  total  of  68  starter,  and  67  generator,  repair  proficiency  tests  were 
administered  to  70  different  subjects  distributed  across  the  10  JE  categories  as 
shown  in  Table  2.  For  subjects  taking  both  tests,  test  order  was  randomly  varied 
across  subjects  to  control  for  sequence  effects.  Subjects  who  had  different 
exposures  to  starter  and  generator  repairs  appeared  in  different  JE  categories  in 
the  two  tests.  For  example,  a  mechanic  who  had  performed  two  starter  and  four 
generator  repairs  would  be  placed  in  JE  Categories  2  and  4,  respectively,  for  the 
starter  installation  and  generator  installation  tests. 


TABLE  2 


NUMBER  OF  MECHANICS  TESTED  IN  EACH  JOB  EXPOSURE  CATEGORY 


JOB  EXPOSURE 
CATEGORY 


REPAIR  TEST 

STARTER  TEST  GENERATOR  TEST  TOTAL 


TESTING  CONDITIONS 

Testing  was  conducted  between  26  July  and  19  October  1982  in  the  2/34 
Battalion  welding  shop  at  Fort  Carson,  Colorado.  The  shop  was  off  limits  to 
maintenance  personnel  during  testing  periods,  with  the  shop  doors  always  closed  to 
provide  subjects  with  a  non-distracting  test  environment.  Overhead  illumination 
was  good. 

Both  tests  were  conducted  on  an  M60A1  engine  that  had  been  removed  from 
the  tank  hull  and  placed  on  an  engine  stand.  Prior  to  the  study,  the  engine  and  all 
test  replacement  parts  were  steam-deaned.  To  ensure  comparable  test  conditions 
across  subjects,  all  tools,  parts,  and  technical  manuals  (TMs)  were  placed  in  fixed 
locations  on  a  plastic-coated  display  board  situated  near  the  engine.  Use  of  the 
display  board  eliminated  the  need  for  men  to  rummage  through  tool  boxes  or  find 
damaged  TMs  and  parts,  activities  that  would  contaminate  the  scoring  of  the  test 
components. 


JOB  EXPOSURE  INDEX 


The  JE  level  of  each  subject  was  defined  via  interview.  Upon  arriving  at  the 
test  site,  a  subject  was  asked  to  indicate  the  number  of  times  he  had  replaced  a 
tank  starter,  and  the  number  of  times  he  had  replaced  a  tank  generator,  since 
completing  formal  school  training.  Subjects  were  required  to  give  their  JE  as  a 
single  number,  not  a  range  of  numbers.  Subjects  were  placed  in  the  JE  category 
that  corresponded  to  their  self-reported  JE. 

TEST  INSTRUMENT 

Starter  installation  and  generator  installation  were  each  broken  down  into  a 
sequence  of  discrete,  observable  performance  steps  that  could  be  objectively 
scored  by  an  experienced  maintenance  technician.  Based  on  input  from  Fort 
Carson  maintenance  experts,  and  an  analysis  of  the  prescribed  steps  in  TM  9-2350- 
257-20-1-3,  preliminary  versions  of  each  test  instrument  were  field-tested  and 
revised.  The  final  versions  of  the  instruments  are  presented  in  Appendix  A 
(starter)  and  Appendix  B  (generator).  Because  time  and  resources  were  limited,  the 
removal  steps  and  the  final  "ground-hop"  check-out  for  each  task  were  not  included 
in  the  test.  A  complete  description  of  all  parts,  tools,  manuals,  and  scoring  rules 
formed  part  of  each  test  instrument. 

Maximum  score  on  each  proficiency  test  was  100.  The  toted  score  on  each 
test  was  the  sum  of  scores  on  six  subtests  as  shown  in  Table  3.  The  rationale 
behind  the  scoring  system  is  described  below. 

Knowledge  of  Technical  Manual 

The  test  began  with  10  questions,  worth  1  point  each,  that  covered 
information  contained  in  the  TM  provided  for  that  repair.  Subjects  were 
encouraged,  though  not  required,  to  use  the  TM  to  answer  the  questions.  Since 
most  subjects  elected  to  use  the  TM,  these  questions  were  designed  to  test  the 
man's  ability  to  use  the  TM  rather  than  his  memory  of  task-related  facts. 
Performance  was  not  timed. 


TABLE  3 


MAXIMUM  NUMBER  OF  POINTS  AWARDED  IN  EACH  SUBTEST 


REPAIR  TEST 

GENERAL  CATEGORY 

SUBTEST 

STARTER 

GENERATOR 

Accuracy 

Knowledge  of  TM 

10 

10 

Accuracy 

Following  Sequence 

zi 

30 

Accuracy 

Selecting  Correct  Tool 

11 

12 

Accuracy 

Using  Tool  Correctly 

11 

12 

Accuracy 

Checking  Quality 
(Working  to  Spec.) 

12 

3 

Time 

Time  to  Finish  (min): 

If  under  40  award: 

If  between  41-48,  award: 
If  between  49-55,  award: 
If  between  56-60,  award: 
If  over  60,  award: 

25 

18  use 
11  Vonly 
4  one 

0  ) 

33 

24  use 

15  Vonly 

6  one 

0  J 

Proficiency 

Total  Possible 

100 

100 

Following  Correct  Step  Sequence 

The  hands-on  component  of  the  test  consisted  of  a  series  of  discrete, 
objective  performance  steps — 31  for  the  starter  test  and  30  for  the  generator  test. 
Subjects  were  awarded  a  point  for  every  step  performed  in  the  correct  sequence. 
In  the  case  where  a  group  of  steps  was  either  left  out  or  performed  out  of 
sequence,  points  were  lost  only  in  the  steps  directly  affected  by  the  sequence 
violations. 


Selection  and  Use  of  Tools 

Eleven  steps  in  the  starter  test  and  12  in  the  generator  test  involved  tools. 
For  these  steps,  the  test  administrator  could  award  up  to  two  additional  points  if 
the  mechanic  (1)  selected  the  correct  tool  and,  (2)  used  that  tool  correctly.  For 
example,  consider  TM  Step  Number  18  of  the  Generator  test,  "tighten  screws  using 
9/16  in.  socket."  The  subject  would  lose  one  point  if  he  had  selected  the  wrong 
tool,  such  as  a  9/16"  open  end  wrench.  Another  point  would  be  lost  if  he  used  the 
wrench  as  a  crowbar,  rather  than  to  tighten  a  nut. 

Checking  Quality  of  Key  Steps  (Working  to  Specifications) 

Some  steps  were  especially  critical  to  the  success  of  the  repair,  and  their 
performance  had  to  be  checked  against  a  known  standard.  Twelve  steps  in  the 
starter  test  and  three  in  the  generator  test  required  an  explicit  check  for  quality. 
In  some  of  these  steps,  the  subject  had  to  work  to  a  certain  specification,  such  as 
by  tightening  a  locknut  to  a  given  torque.  In  others,  he  had  to  visually  inspect  his 
work,  as  when  ensuring  that  a  terminal  nut  was  clean. 

Time 

Time  to  complete  the  performance  steps  was  recorded  to  the  nearest 
minute.  A  total  proficiency  score  was  derived  by  converting  time  in  minutes  into 
bonus  points  according  to  the  conversion  rule  given  in  the  lower  part  of  Table  3. 
These  bonus  points  were  then  added  to  the  other  criteria  scores  to  yield  the  total 
proficiency  score.  The  upper  and  lower  time  cutoffs,  60  min.  and  40  min.,  were 
based  on  what  Fort  Carson  maintenance  experts  considered  to  represent  slow  and 
fast  installation  times,  respectively. 

TEST  PROCEDURE 

The  test  site  was  secured  and  the  test  materials  assembled  during  the  week 
before  data  collection  started.  The  test  administrator,  a  recently  retired  battalion 
maintenance  specialist  with  over  20  years  of  repair  experience,  supervised  the 
installation  of  the  test  engine  and  readied  it  for  testing.  The  authors  and  the  test 
administrator  met  often  during  the  preparatory  week  to  discuss  principles  of  good 
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test  administration,  the  importance  of  tight  control  over  test  conditions,  and  the 
rules  for  providing  feedback  to  subjects.  The  characteristics  of  these  sessions  were 
summarized  in  an  earlier  report.*  During  the  latter  part  of  the  week,  the  rules  for 
scoring  each  performance  step  were  finalized  and  practice  tests  were  given. 

Subjects  were  drawn  from  four  armored  battalions  and  scheduled  a  week  in 
advance.  Each  subject  was  tested  individually  in  the  following  way:  Upon  arriving 
at  the  test  site,  the  subject  was  asked  to  give  his  name,  primary  MOS,  paygrade, 
and  social  security  number.  Next,  he  read  a  privacy  act  release  form  and  was  told 
that  his  participation  in  the  study  was  strictly  voluntary  (only  one  mechanic 
refused  to  participate).  Reading  from  the  script  shown  in  Appendix  C,  the  test 
administrator  asked  the  subject  for  the  number  of  JEs  he  had  on  each  repair  task, 
as  well  as  an  estimate  of  the  number  of  days  since  he  had  last  performed  each  task. 
The  subject  was  also  asked  for  the  name  of  his  AIT  school  and  when  he  graduated. 
Next,  the  test  administrator  briefed  the  subject  on  the  purpose  of  the  study  and 
showed  him  the  test  materials. 

Testing  began  with  the  TM  questions  for  one  of  the  tests.  Immediately  after 
answering  the  last  question,  the  subject  began  the  hands-on  part  of  the  test  and  the 
test  administrator  started  the  stop  watch.  During  the  performance  of  each  step, 
the  test  administrator  checked  each  scoring  criterion  for  accuracy.  Errors  were 
indicated  on  the  test  in  two  ways:  (1)  by  deducting  a  point  from  the  appropriate 
scoring  criterion,  and  (2)  by  annotating  the  inappropriate  action  in  the  right-hand 
margin.  Once  the  subject  indicated  that  the  repair  was  completed,  his  time  was 
recorded  and  he  was  asked  to  remove  all  replaced  parts  and  return  them  to  the 
display  board.  These  removal  steps  were  not  scored,  but  were  necessary  to  set  up 
the  test  site  for  the  next  subject. 

The  second  test  began  immediately  after  the  parts  from  the  first  test  had 
been  returned.  Once  the  second  test  was  completed,  the  subject  performed  the 
installation  steps  in  reverse  and  returned  all  parts  to  the  display  board  as  before. 
During  this  activity,  the  test  administrator  tallied  the  points  earned  on  each  test. 

1Spiker,  V.  A.  Maintenance  performance  systems  (organizational).  Test  administrator's 
guide  for  MOS  63N  maintenance  proficiency  test.  Santa  Barbara,  CA:  Anacapa 
Sciences,  Inc.  (Technical  Report  465-22),  August  1982. 


The  subject  was  given  his  total  proficiency  score  on  each  test,  as  well  as  each 
subtest  score.  Feedback  on  individual  steps  that  posed  particular  difficulties  was 
also  given  to  the  subjects.  Total  administration  time  for  both  tests  ranged  from 
two  to  three  hours. 

The  collected  data  were  coded,  keypunched,  and  stored  on  an  NASCO  AS/6 
computer  at  the  University  of  California,  Santa  Barbara.  The  data  were 
subsequently  processed  using  the  Statistical  Analysis  System.* 


*Helwig,  J.  T.,  <5c  Council,  K.  A.  SAS  user's  guide,  1979  edition.  Cary,  North 
Carolina:  SAS  Institute,  Inc.,  1979. 
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RESULTS 


The  study  provided  answers  to  four  specific  research  questions: 

•  How  closely  related  are  JE  and  tested  maintenance  proficiency?  JE  was 

strongly  correlated  with  maintenance  proficiency,  accounting  for  one-half 
of  the  total  variation  in  the  test  data. 

•  What  is  the  nature  of  the  relationship  between  JE  and  maintenance 
proficiency?  Proficiency  increases  sharply  with  JE,  peaks  between  5  and 
6  exposures,  and  declines  slightly  with  exposures  beyond  6  on  the  tasks 
tested. 

•  How  generalizable  is  the  relationship?  Comparable  results  were  obtained 
for  the  two  different  maintenance  tasks,  suggesting  that  the  functional 
relationship  is  reliable  and  generalizable  to  tasks  of  this  type.  However, 
starter  and  generator  installation  are  but  two  of  the  many  corrective 
maintenance  tasks  performed  by  the  63N  mechanic. 

•  Does  JE  reflect  the  various  dimensions  of  maintenance  work?  Similar 
functional  relationships  were  found  between  JE  and  measures  of  six 
different  aspects  of  maintenance  work.  These  results  suggest  that  JE 
reflects  different  components  of  maintenance  proficiency — quality,  pro¬ 
cedural,  time,  information  use,  and  tool  selection  and  use. 

Mean  test  scores  for  every  combination  of  scoring  criterion,  task,  and  JE 
category  are  summarized  in  Appendix  D.  The  methods  used  to  pool  and  analyze  the 
data  are  described  in  Appendix  E. 

Results  of  the  analysis  of  the  overall  proficiency  data,  and  the  data  from 
the  individual  subtests,  are  described  in  the  sections  below. 


EFFECT  OF  JOB  EXPOSURE  ON  OVERALL  MAINTENANCE  PROFICIENCY 

The  relationship  between  JE  and  proficiency  was  quite  strong,  with  JE 
accounting  for  approximately  one-half  the  variance  in  the  data  in  each  test. 
Overall  mean  tested  proficiency  plotted  as  a  function  of  JE  for  the  starter  and 
generator  tests  separately  is  shown  in  Figure  2.  The  JE  effect  easily  reached 
statistical  significance  in  both  tests  (p<  .01). 


Figure  2.  Effect  of  job  exposure  on  overall  repair  proficiency. 

The  JE  curves  for  both  tests  are  characterized  by  proficiency  rising 
markedly  over  the  lower  range  of  JE  values,  peaking  at  5  to  6  exposures,  and  then 
declining  slightly.  Overall  test  performance  was  relatively  good,  with  only  the 
subjects  who  had  not  previously  performed  the  task  scoring  below  80.  Overall 
proficiency  was  consistently  higher  on  the  generator  test,  mainly  because  the 
starter  required  less  time  to  install. 

EFFECT  OF  JOB  EXPOSURE  ON  REPAIR  TIME 

Mean  time  to  complete  the  test  decreased  generally  with  JE,  reached  a 
minimum,  and  then  increased  slightly  at  the  greatest  JE  categories  for  both  starter 
and  generator  repair  tasks.  These  results  are  shown  in  Figure  3.  The  dedine  in 
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JOB  EXPOSURE  CATEGORY 

Figure  3.  Effect  of  job  exposure  on  time  to  complete  test. 

mean  time  across  JEs  implies  more  efficient  performance  and  is  thus  consistent 
with  the  results  for  total  proficiency.  The  JE  effect  on  the  time  to  install  the 
starter  was  especially  strong,  accounting  for  almost  half  the  total  data  variance. 
A  weaker  relationship  was  found  in  the  generator  data,  with  JE  accounting  for  34% 
of  the  test  data  variance. 

Consistent  with  the  proficiency  data,  the  time  curves  for  the  two  tests  are 
roughly  parallel,  with  the  starter  taking  15-20  fewer  minutes  to  install  than  the 
generator.  For  both  tests,  the  decrease  in  repair  time  over  the  lower  JE  values  and 
the  slight  increase  at  the  highest  JE  category  were  statistically  significant. 
Fastest  starter  installation  times  were  recorded  by  mechanics  in  JE  Category  7-8, 
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with  a  mean  slightly  under  31  minutes.  Generator  installation  was  fastest  for 
mechanics  in  JE  Category  5-6,  with  a  mean  time  slightly  over  18  minutes. 


EFFECT  OF  JOB  EXPOSURE  ON  HEP  AIR  ACCURACY 

Similar  results  were  obtained  from  an  aggregate  score  of  repair  accuracy  as 
were  obtained  from  the  individual  subtest  scores  of:  TM  knowledge,  following  the 
prescribed  step  sequence,  tool  selection/use,  and  quality  check  of  key  performance 
steps.  To  allow  the  scores  from  the  two  tests  to  be  plotted  on  the  same  scale, 
accuracy  was  converted  to  a  percentage  by  dividing  the  summed  score  by  the 
maximum  points  possible,  75  for  the  starter  test  and  67  for  the  generator  test,  and 
multiplying  by  100.  Percent  total  accuracy  is  plotted  as  a  function  of  JE  in 
Figure  4. 
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JOB  EXPOSURE  CATEGORY 

Figure  4.  Effect  of  job  exposure  on  total  repair  accuracy. 


A  strong  functional  relationship  was  found  between  JE  and  repair  accuracy,  similar  to 
the  relationship  between  JE  and  overall  repair  proficiency.  In  both  tests,  the  JE  curve  has  a 
significant  linear  rise,  a  peak  at  JE  Category  5-6,  followed  by  a  significant  decline  in 
performance  for  the  two  highest  JE  categories.  Note  that  repair  accuracy  was  comparable 
on  the  two  tests,  indicating  that  the  better  overall  proficiency  observed  in  the  starter  test 
was  due  to  speed  rather  than  accuracy. 

The  relationship  between  JE  and  repair  accuracy  was  examined  further  by  separately 
analyzing  the  data  for  each  individual  accuracy  measure.  In  general,  these  individual 
analyses  support  the  conclusions  based  on  the  analysis  of  total  repair  accuracy. 

Effect  of  Job  Exposure  on  TM  Knowledge 

Average  scores  on  the  10  TM  questions  were  converted  to  percentage  form  and 
displayed  in  Figure  5.  Consistent  with  the  analysis  of  total  accuracy,  subjects'  TM 
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Figure  5.  Effect  of  job  exposure  on  knowledge  of  technical  manual. 


performance  improved  as  a  function  of  JE,  reaching  a  peak  at  JE  Category  7-8  on  the 
starter  test  and  5-6  in  the  generator  test.  Performance  declined  in  different  JE  categories 
for  the  two  tests.  Mean  starter  test  performance  dropped  by  5%  for  mechanics  in  the 
highest  JE  category;  generator  TM  performance  decreased  for  subjects  in  the  7-8  category 
but  then  rose  4%  for  the  most  experienced  mechanics. 

Effect  of  Job  Exposure  on  Following  Correct  Step  Sequence 

The  percentage  of  performance  steps  correctly  followed  is  plotted  as  a  function  of  JE 
in  Figure  6.  Like  the  previous  analyses  of  accuracy,  the  sequence  criterion  shows  a 
significant  increase,  a  peak,  and  then  a  significant  decrease  over  JE.  Sequence  performance 
was  comparable  between  the  two  tests,  with  maximum  performance  observed  in  JE 
Category  7-8  for  the  starter  test  and  5-6  for  the  generator  test.  Performance  within  a  JE 
category  was  more  variable  than  in  other  measures,  as  reflected  in  the  finding  that  JE 
accounted  for  only  20%  of  the  data  variance  for  each  test. 


Figure  6.  Effect  of  job  exposure  on  following  correct  step  sequence. 
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Effect  of  Job  Exposure  on  Tool  Use/Selection 

The  combined  tool  selection/use  criterion  was  converted  to  a  percentage 
and  plotted  over  JE  categories  as  shown  in  Figure  7.  The  statistically  significant 
proficiency  growth  with  JE  peaked  at  JE  category  7-8  for  both  tests.  The  average 
tool  selection/use  score  was  consistently  higher  in  the  starter  test,  though  not  by 
much.  The  biggest  improvement  in  tool  performance  was  noted  between  the  0  and 
1-2  categories,  increasing  by  5%  in  the  starter  test  and  11%  for  the  generator  test. 


STARTER 

GENERATOR 
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Figure  7.  Effect  of  job  exposure  on  tool  selection/use. 
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The  size  of  these  increases  supports  the  notion  that  proper  tool  selection  and  use  is 
a  primary  skill  component  that  mechanics  should  acquire  early  in  their  career. 

Effect  of  Job  Exposure  on  Checking  the  Quality  of  Key  Steps 

Number  of  key  steps  correctly  checked  for  quality  was  converted  to  a 
percent  and  plotted  over  JE  as  shown  in  Figure  8.  The  two  JE  curves  diverge 
markedly,  requiring  that  the  results  from  the  two  tests  be  discussed  separately. 
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Figure  8.  Effect  of  job  exposure  on  checking  quality  of  key  steps. 


Considering  the  starter  test  first,  performance  showed  a  systematic  and  significant 
increase  over  the  lower  JE  categories,  a  peak  at  5-6,  and  a  significant  decline  in 
the  two  highest  categories.  Performance  within  categories  was  variable,  with  JE 
explaining  only  slightly  more  than  20%  of  the  total  variance. 

The  percentage  of  key  generator  installation  steps  checked  for  quality 
varied  substantially  between  the  low  and  high  JE  categories.  In  particular,  the  15% 
drop  between  the  5-6  and  7-8  JE  categories  placed  the  performance  of  the  most 
experienced  mechanics  at  levels  comparable  to  those  of  beginners.  This 

equivalence  made  the  overall  effect  of  JE  on  quality  check  nonsignificant, 
invalidating  further  statistical  testing  of  individual  mean  differences. 

The  generator  data  should  be  interpreted  cautiously,  however,  since  quality 
scores  were  based  on  only  three  steps.  While  it  is  safe  to  conclude  that 

performance  by  the  most  experienced  mechanics  is  lower  than  that  of  moderately 

experienced  mechanics,  the  observed  decline  to  a  level  equal  to  the  beginning 

mechanic  may  be  exaggerated  because  of  the  small  sample  of  steps  contained  in 
the  criterion. 


CONCLUSIONS  AND  IMPLICATIONS 


A  strong  positive  relationship  was  found  between  JE  and  maintenance  pro¬ 
ficiency,  in  both  repair  accuracy  and  repair  speed.  These  results  suggest  that  JE  is 
a  valid,  easy-to-colleet  indicator  of  the  skill  development  of  maintenance  per¬ 
sonnel.  As  a  consequence,  JE  is  likely  to  be  a  useful  index  for  monitoring 
individual  skill  growth  and  identifying  skill  development  needs. 

IMPLICATIONS  FOR  MPS(0) 

The  strong,  reliable,  generalizable  relationship  between  JE  and  tested 
maintenance  proficiency  supports  the  use  of  JE  for  monitoring  mechanic  skill 
growth  and  targeting  OJT  in  the  MPS(O).  The  JE  is  likely  to  provide  meaningful 
profiles  of  the  task  proficiency  of  individual  mechanics,  and  useful  indices  of 
mechanic  skill  growth  within  units.  Mechanics  with  little  or  no  JE  on  a  given  task 
are  likely  to  be  most  deficient  in  repair  skill,  most  in  need  of  OJT,  and  most  likely 
to  benefit  from  supervised  practice.  As  JEs  accrue,  the  maintenance  proficiency 
of  the  unit  should  also  grow. 

IMPLICATIONS  FOR  OJT 

It  is  worth  noting  that  many  subjects  praised  the  test  conditions  as  providing 
a  good  training  vehicle  for  the  two  repair  tasks.  These  compliments  were 
amplified  by  the  mechanics'  supervisors.  The  opportunity  to  perform  repairs  on 
real  equipment,  under  quiet  conditions,  and  in  the  presence  of  immediate  feedback 
from  an  expert  prompted  phrases  like  "its  a  great  way  to  learn"  and  "sure  wish  Fd 
had  this  in  AIT." 

The  test  conditions  employed  in  this  study  could  be  readily  adapted  for  use 
in  OJT.  For  example,  mechanics  having  few  JEs  on  a  given  task  might  be  singled 
out  for  performance  testing  under  "OJT  conditions."  These  conditions  would 
consist  of  the  requisite  repair  equipment  (e.g.,  a  detached  tank  engine  if  the  task 
required  it),  TMs,  tools,  and  repair  parts.  All  materials  would  be  housed  in  a 
secure,  enclosed  area  of  the  motor  pool.  OJT  candidates  would  be  formally 
scheduled  for  the  test  in  advance,  and  would  perform  the  repair  under  the  watchful 


i 


eye  of  a  senior  maintenance  technician.  Immediate  and  detailed  feedback  on  each 
performance  step  would  be  given.  Besides  providing  task-related  feedback,  the 
OJT  session  would  also  cover:  how  to  read  a  TM,  the  general  principles  governing 
proper  tool  use,  and  the  importance  of  working  to  specifications.  To  the  extent 
that  company  training  managers  fostered  the  motivation  for,  and  garnered  the 
resources  to  sustain,  such  a  program,  the  overall  level  of  repair  proficiency  would 
likely  increase  rapidly. 

IMPLICATIONS  FOR  FURTHER  RESEARCH 

A  single  study  cannot  possibly  resolve  every  major  issue  in  its  area.  In  the 
present  context,  additional  research  pertinent  to  the  relationship  between  JE  and 
repair  skill  is  desirable  on  two  fronts: 

The  JE-Proficiency  Relationship  Should  be  Examined  for 
Other  Repair  Tasks  and  Other  MOSs 

Can  the  conclusions  drawn  from  these  data  be  applied  to  other  63N  repair 
tasks  and  to  tasks  performed  by  other  mechanic  MOSs?  Starter  and  generator 
installation  are  representative  of  many  repairs  performed  at  the  organizational 
level,  but  other  qualitatively  different  repair  tasks  are  performed  as  well.  For 
instance,  would  the  same  function  be  obtained  on  an  easier  task,  such  as 
adjust/replace  minor  components;  on  a  more  complex  task,  like  troubleshooting/ 
diagnosis?  Similar  questions  regarding  generalizability  should  also  be  addressed  for 
other  mechanic  MOSs,  such  as  turret  mechanics  (45N),  radio  repairmen  (31V),  or 
wheeled  vehicle  repairmen  (63B/S).  Studies  using  a  design  similar  to  this  one  could 
be  conducted  on  other  tasks  and  MOSs  to  shed  light  on  the  issue  of  generalizability. 

The  JE-Proficiency  Relationship  for  Mechanics  Having  10  or  More 
JEs  Should  be  Examined  Further 

How  reliable  and  widespread  is  the  less-than-optimal  repair  proficiency  of 
the  most  experienced  subjects?  Only  28  such  mechanics  were  included  in  the 
present  study,  so  a  full-scale  repeat  of  the  study  on  a  larger  group  of  experienced 
mechanics  is  needed.  The  study  should  be  conducted  at  another  site  to  determine 
if  the  effect  described  in  this  report  is  unique  to  Fort  Carson.  If  it  is  not,  then  the 
major  role  that  senior  mechanics  should  play  in  future  OJT  efforts  may  have  to  be 


redefined.  Finally,  future  research  should  also  examine  the  contribution  of  specific 
factors  to  this  drop  in  repair  proficiency,  such  as  lack  of  motivation,  out-dated 
technical  knowledge,  false  confidence  in  the  mechanic's  own  ability,  and 
unwillingness  to  retrieve  and  consult  appropriate  technical  manuals. 


APPENDIX  A 

TEST  (M60  CM3-63N)  -  INSTALL  A  STARTER 
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ORGANIZATIONAL  MAINTENANCE 
TEST  ADMINISTRATOR’S  GUIDE 

MOS  63N  MAINTENANCE  PROFICIENCY  TEST 


M60  CMS  -  63N 


OBJECTIVE 

•  TASK 

-  Install  a  starter  on  a  2C  M60A1  RISE  Passive  tank  engine  (includes 
installation  of  starter  low  voltage  relay  solenoid). 

•  CONDITIONS 

-  Powerplant  removed  from  M60A1  RISE  Passive  tank 

-  Starter  (and  solenoid)  removed  from  powerpack  -Cables  already  tagged 
To  be  available  on-site: 

-  TM  9-2350-257-20-1  Vols.  1  through  4 
—  One  assistant 

—  63N  mechanic's  tool  kit 
—  A  copy  of  the  test 

—  All  tools  and  supplies  listed  on  page  10-23  and  page  10-30,  TM 
9-2350-257-20-1-3 

—  Replacement  starter 

•  INFORMATION  FOR  ADMINISTRATOR  ONLY 

-  This  task  will  require  an  assistant  mechanic. 

•  STANDARD 

-  Task  will  be  completed  in  60  minutes. 

-  TM  references  for  sequence,  tolerances  will  be  followed,  if  available. 

-  Safety  precautions  will  be  observed. 
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M60  CM3  -  63N 

- - i 


ADMINISTRATOR'S  NOTES 
Background 

•  You  must  know  this  task  before  giving  the  test.  The  task  is  explained  in  detail 
in  pages  10-27  through  10-29,  TM  9-2350-257-20-1-3.  Steps  1  to  14  on  pages 
10-27  through  10-29  and  steps  1  and  2  on  page  10-31  constitute  the  material  in 
the  test. 

•  You  will  need  a  stopwatch  or  wristwatch. 

•  Interfere  with  the  test  only  if  a  hazardous  situation  is  developing. 

Immediately  Before  the  Test 

•  Check  that  the  man  who  reports  for  the  test  has  been  assigned  by  Anacapa. 

•  Emphasize  the  purpose  of  the  test  is  to  help  find  out  what  additional  skill 
training  is  needed  that  will  further  his  career  and  improve  maintenance. 

•  Read  him  the  task,  condition  and  standards. 

•  Ask  the  examinee  for  the  number  of  times  he  has  performed  this  task  since 
AIT.  Record  on  page  d. 

•  Ask  the  examinee  how  long  it  has  been  since  he  last  performed  this  task  on  the 
line,  or  since  AIT  if  he  has  no  previous  line  experiences  for  this  task.  Record 
on  page  d. 

•  Tell  him  you'll  review  his  results  with  him  immediately  after  the  test. 

•  Explain  that  the  test  is  in  two  parts: 

-  Part  1  -  A  short  test  (10  questions)  of  general  knowledge  related  to  the 
task 

-  Part  2  -  A  "hands-on"  demonstration  of  the  task 

•  Explain  that  he  may  use  any  of  the  reference  materials  provided  in  any  way  he 
feels  will  help  him  complete  the  test. 

•  Explain  that  the  10  questions  in  Part  1  are  not  timed. 

During  the  Test 

•  Now  ask  him  the  questions  in  Part  1  and  record  his  score.  (The  test  question 
sheets  are  also  your  score  sheets.) 

•  Explain  that  he  will  now  do  the  task  in  Part  2. 

•  Tell  him  he  has  60  minutes  to  do  the  task  but  that  you  won't  start  timing  till  he 
says  he  is  ready. 

•  Start  timing  on  his  signal  and  record  his  scores. 

•  If  an  unavoidable  distraction  arises  during  the  test,  temporarily  suspend  timing 
until  the  distraction  is  taken  care  of.  Then,  resume  timing. 

•  Constantly  safeguard  the  mechanic  against  hazardous  situations. 

•  Stop  timing  when  he  says  the  job  is  finished. 

After  the  Test 

•  Compute  the  score. 

•  Explain  the  scoring  system  to  the  mechanic  and  review  the  results  with  him. 

•  Correct  any  procedural  or  quality  errors,  and  safety  violations  at  this  point. 

•  Have  the  mechanic  sign  the  Test  Grade  Computation  Sheet  agreeing  he  has 
received  feedback. 
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If  60  CM3  -  6: 


TEST  SCORING  NOTES* 

Part  1  (Not  timed)  is  scored  by  awarding  1  point  for  each  correct  answer. 
("Correct  answers"  may  be  given  orally,  or  by  the  mechanic  pointing  to  the 
manual  or  part,  related  to  the  test  question.) 

Part  2  (Timed— 60  minutes  allowed)  is  scored  by  awarding  1  point  each  for: 

-  Maintaining  the  performance  step  sequence** 

-  Selecting  the  correct  tool(s) 

-  Using  tool(s)  properly 

-  Working  to  specifications/tolerances 
To  Compute  the  Test  Grade  for  Part  2 

Add  across  each  row  by  page  to  get  score  for  each  performance  component. 

Add  each  column  to  find  mechanic's  total  scores  on  tool  selection  and  use, 
working  to  specs,  etc. 

Add  totals  across  bottom  row,  page  4,  to  get  overall  test  score. 

If  Part  2  is  completed  with  an  overall  score  of  52  within  the  time  blocks 
shown,  add  the  incremental  points  as  shown  in  the  table  below.  If  Part  2  is 
completed  but  outside  the  time  limits,  no  incremental  points  are  awarded. 

Incremental  Points  for  Completion  to  Time  Standard 

If  mechanic  has  a  score  of  at  least  52  on  Part  2 

AND 

-  Has  completed  the  test  in  under  40  minutes-add  25  points  to  his  score 

-  Has  completed  the  test  in  41-48  minutes-add  18  points  to  his  score 

-  Has  completed  the  test  in  49-55  minutes-add  11  points  to  his  score 

-  Has  completed  the  test  in  56-60  minutes-add  4  points  to  his  score 

-  Has  completed  the  test  in  over  60  minutes-add  0  points  to  his  score 


*A  computation  sheet  is  provided  on  page  d. 

**The  TM  will  be  the  reference  authority  for  sequence,  tools,  and  specifications, 
if  appropriate.  If  not,  use  local  standards  and  procedures. 
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TEST  GRADE  COMPUTATION  SHEET 


| 

M60  CM3  -  63N  | 


Possible 

Points 

Actual 

Points 

Part  1-10  points 

10 

Part  2 

Done  in  correct  sequence  - 

31 

Selection  of  correct  tool  - 

11 

Correct  use  of  tool  - 

11 

►65 

► - 

Working  to  correct  specs  - 

12  J 

Time  for  Performance 

If  score  for  Part  2  =  58  and  over 

and  Performance  is  under  40  minutes, 
(full  credit) 

add  25  -v 

Performance  in  41-48  minutes  add 
(partial  credit) 

Performance  in  49-55  minutes  add 
(partial  credit) 

18 

11 

►  25 

Select 

only 

one 

- - 

Performance  in  56-60  minutes  add 
(partial  credit) 

4 

Performance  60  minutes  add 
(no  credit) 

0- 

J 

Total 

Possible  = 
Points 

100 

Actual 
Points : 
Scored 

= 

TESTEE  DATA 


Date 


1 

2. 

3. 

4. 

5. 

6. 


Name 


SSN 


Primary  MOS 
AIT 

Number  of  Prior  Task  Experiences 
Number  of  Days  Since  Last  Performance 

Finished 


Time  Taken  for  Test 


Started 


•  Received  debriefing  X 
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M60  CMS  -  63N 


PART  1:  GENERAL  BACKGROUND  KNOWLEDGE  RELATED  TO  TM  USE  IN  TEST 

(one  point  for  each  correct  answer) 


Points 


*1.  What  current  TM  series  has  organizational  maintenance  infor¬ 
mation  for  the  M60A1  RISE  Passive  2C  engine? 

ANS:  TM  9-2350-257-20  series  _ 

2.  What  is  the  date  of  the  latest  revision? 

ANS:  January  1981  _ 

3.  Which  volume  of  the  TM  tells  how  to  remove  and  install  the 
starter? 

ANS:  TM  9-2350-257-20-1-3  (Volume  3)  _ 

4.  Is  a  special  tool  required  to  remove  and  install  the  starter? 

ANS:  Yes  (see  special  tools,  page  10-23,  TM  9-2350-257-20-1-3 

or  TM  9-2350-257-20-1-1,  page  3-1)  _ 

5.  Show  me  (in  the  TM)  where  it  notes  how  much  gap  is  allowed 
between  the  starter  case  and  support  cradle  assembly. 

ANS:  Page  10-28,  Step  9,  TM  9-2350-257-20-1-3 

(NOTE,  no  gap  allowed)  _ 

6.  Show  me  (in  the  TM)  how  much  torque  pressure  is  needed 
for  the  nuts  holding  the  cables  to  the  large  terminal 

of  the  starter  motor. 

ANS:  Page  10-29,  Step  13,  TM  (as  above) 

(168-192  lb-in  or  19-22  N  m)  _ 

7.  What  are  the  slip  joint  pliers  used  for  on  this  job? 

ANS:  To  fit  lockwire  on  screws  holding  starter 

cradle  and  bracket  to  oil  pan.  _ 

8.  What  goes  between  the  starter  and  the  starter  drive 
adaptor? 

ANS:  A  gasket.  _ 

9.  What  special  tool  is  used  to  tighten  the  self-locking 
nuts  on  the  starter  drive  adaptor  bolts? 

ANS:  Special  wrench  _ 

10.  When  do  you  install  the  starter  low  voltage  relay  solenoid? 

ANS:  After  the  starter  is  installed  and  connected  _ 

(Total  Possible  Points  =  10) 

Total  Actual  Points  = 

*  Ask  "Have  you  taken  any  previous  NAST(I)'s  for  this  equipment?”  If  YES  -  Award 
2  points  and  go  to  3.  If  NO  -  Start  at  1. 
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M60  CM3  -  63N 


Standard 


Install  new  gasket  on 
starter  drive  adapter. 

Ease  starter  into 
starter  drive  adapter. 

(Support  starter)  install 
new  self-locking  nut 
on  upper  bolt  of  drive 
adapter. 

Hold  the  bolt  with 
a  15/16"  wrench  and 
tighten  nut  with 
special  wrench. 

Install  a  new  self-locking 
nut  on  the  bottom 
(outboard)  bolt  of 
drive  adapter. 

Hold  the  bottom  (out¬ 
board  bolt  with  a 
15/16"  wrench  and 
tighten  nut  with 
special  wrench. 

With  fingers,  start  a  new 
self-locking  nut  on 
the  bottom  (inboard) 
bolt  of  drive  adapter. 

Tighten  nut  with  special 
wrench. 

5.  Slide  cradle  assembly  into 
position  on  starter. 

6.  With  fingers,  install  three 
screws  (bolts)  and  flat 
washers  to  secure  starter 
cradle  assembly  bracket 
to  engine  oil  pan. 


Possible  Subtotals 

Actual  Subtotals 


•Steps  1-14,  pages  10-27  through  10-29,  TM  9-2350-257-20-1-3 
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M60  CM3  -  63N 


T 

i» 

I 

I 

I 

I 

I 


Derived 

from 

TM# 

Performance 
Component  Step 

Done  in 
Correct 
Sequence 

Selection 
of  Cor¬ 
rect  Tool 

Correct 
Use  of 
Tool 

Checking 

Quality 

Against 

Standard 

Total 

Possible  Comments 

7a. 

With  fingers,  install 
double-braided  ground 
straps  with  screw  (bolt) 
and  flat  washer. 

1 

1 

b. 

Ensuring  flat  washer  seats 
flat  against  bracket 
face,  install  screw  (bolt) 
to  starter  cradle  assembly 
bracket  and  engine  oil 
pan. 

1 

1 

8a. 

Tighten  self-locking 
nuts  on  U-bolts  evenly 
with  1/2"  wrench. 

1 

1 

1 

1 

4 

b. 

Ensure  U-bolts  are  seated 
properly. 

1 

- 

- 

1 

2 

9a. 

Tighten  the  four  screws 
(bolts)  and  flat  washers 
securing  the  starter  cra¬ 
dle  assembly  bracket  to 
engine  oil  pan  with  9/16" 
socket  and  extension 

1 

1 

1 

3 

b. 

Ensure  there  is  no  gap 
between  starter  case  and 
cradle  assembly. 

1 

— 

- 

1 

2 

c. 

Adjust  position  of 
cradle  assembly,  if 
needed. (1) 

1 

— 

1 

2 

10a. 

Install  low  voltage 
relay  solenoid  bracket 
to  engine  oil  pan  with 
four  screws  and  lock 
washer 

1 

| 

1 

2 

b. 

Tighten  the  screws 
with  i"  socket  and 
extension. 

1 

1 

1 

3 

Possible  Subtotals 

Actual  Subtotals 

9 

3 

3 

5 

20 

(l)  If  cradle  assembly  does  not  need  adjusting  award  2  points. 
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M60  CM3  -  63N 


Derived 

from 

TM# 

Performance 
Component  Step 

Done  in 
Correct 
Sequence 

Selection 
of  Cor¬ 
rect  Tool 

Correct 
Use  of 
Tool 

Checking 

Quality 

Against 

Standard 

Total 

Possible  Comments 

11. 

With  screwdriver,  connect 
small  cable  to  bottom 
front  of  starter  switch 
solenoid  relay  with  screw 
and  external  tooth 
lockwasher. 

1 

1 

1 

1 

4 

12a. 

Connect  heavy  cables 
and  small  cable  to 
bottom  terminal  of 
starter  switch  relay 
solenoid  with  nut  and 
lockwashers. 

1 

1 

b. 

Tighten  nut  to  168- 
192  lb-in  (19-22  N  m) 
with  3/4"  socket  and 
torque  wrench.* 

1 

1 

1 

1 

4 

13a. 

Connect  double-braided 
ground  straps,  heavy 
cables,  and  small  cable 
to  large  terminal  with 
nut  and  lockwashers. 

1 

1 

b. 

Tighten  nut  to  168- 
192  lb-in  (19-22  N  m) 
with  3/4"  socket  and 
torque  wrench.* 

1 

1 

1 

1 

4 

(Now  go  to  page  10-31  for  performance  step  breakdown) 

la. 

Place  insulators  between 
low  voltage  relay 
solenoid  and  engine 
mounting  bracket. 

1 

*“  j 

“ 

1 

2 

b. 

Place  LV  relay  solenoid 
over  mounting  bracket 
on  insulators  with 
electrical  plug  facing 
front. 

1 

1 

2 

c. 

Secure  solenoid  to 
plate  with  screws,  arid 
new  self-locking  nuts. 

1 

| 

1 

d. 

Tighten  with  7/16" 
socket. 

1 

1 

1 

“ 

3 

Possible  Subtotals 

9 

4 

4 

5 

22 

Actual  Subtotals 

■■  . 

-  ■ 

■  .  ■  ■■ 

■ 

•Test  Administrator:  Check  reading. 
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APPENDIX  B 

TEST  (M60  CM4-63N)  -  INSTALL  A  GENERATOR 


If  60  CM4  -  63N 


ORGANIZATIONAL  MAINTENANCE 
TEST  ADMINISTRATOR’S  GUIDE 

MOS  63N  MAINTENANCE  PROFICIENCY  TEST 


OBJECTIVE 
•  TASK 


-  Install  a  generator  on  a  2C  M60A1  RISE  Passive  tank  engine 


•  CONDITIONS 

-  Powerplant  removed  from  M60A1  RISE  Passive  tank 

-  Generator  removed  from  powerpack 
To  be  available  on-site: 

—  Replacement  generator  ready  for  installation,  with  fittings 
and  hose  assemblies  installed 

—  63N  mechanic's  tool  kit 
—  Special  tools 

-  TM  9-2350-257-20-1-1  through  -1-4 
—  One  assistant 

—  A  copy  of  the  test 

•  INFORMATION  FOR  ADMINISTRATOR  ONLY 

-  This  task  will  require  an  assistant  mechanic. 

•  STANDARD 

-  Task  will  be  completed  in  60  minutes. 

-  TM  references  for  sequence,  tolerances  will  be  followed,  if  available. 

-  Safety  precautions  will  be  observed. 
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M60CM4 


ADMINISTRATOR'S  NOTES 

Background 

•  You  must  know  this  task  before  giving  the  test  The  task  is  explained  in  detail 
in  pages  10-9  through  10-11,  TM  9-2350-257-20-1-3.  Steps  12-35  constitute 
the  material  in  the  test. 

•  You  will  need  a  stopwatch  or  wristwatch. 

•  Interfere  with  the  test  only  if  a  hazardous  situation  is  developing. 

Immediately  Before  the  Test 

•  Check  that  the  man  who  reports  for  the  test  has  been  assigned  by  Anacapa. 

•  Emphasize  the  purpose  of  the  test  is  to  help  us  find  out  what  additional  skill 
training  is  needed  that  will  further  his  career  and  improve  maintenance. 

•  Read  him  the  task,  condition  and  standards. 

•  Ask  the  examinee  for  the  number  of  times  he  has  performed  this  task  since 
AIT.  Record  on  page  d. 

•  Ask  the  examinee  how  long  it  has  been  since  he  last  performed  this  task  on 

the  line,  or  since  AIT  if  he  has  no  previous  line  experiences  for  this  task.  Record 
on  page  d. 

•  Tell  him  you'll  review  his  results  with  him  immediately  after  the  test. 

•  Explain  that  the  test  is  in  two  parts: 

-  Part  1  -  A  short  test  (10  questions)  of  general  knowledge  related  to 
the  task 

-  Part  2  -  A  "hands-on"  demonstration  of  the  task 

•  Explain  that  he  may  use  any  of  the  reference  materials  provided  in  any  way 
he  feels  will  help  him  complete  the  test. 

•  Explain  that  the  10  questions  in  Part  1  are  not  timed. 

During  the  Test 

•  Now  ask  him  the  questions  in  Part  1  and  record  his  score.  (The  test  question 
sheets  are  also  your  score  sheets.) 

•  Explain  that  he  will  now  do  the  task  in  Part  2. 

•  Tell  him  he  has  60  minutes  to  do  the  task  but  that  you  won't  start  timing  till 
he  says  he  is  ready. 

•  Start  timing  on  his  signal  and  record  his  scores. 

•  If  an  unavoidable  distraction  arises  during  the  test,  temporarily  suspend  timing 
until  the  distraction  is  taken  care  of.  Then,  resume  timing. 

•  Constantly  safeguard  the  mechanic  against  hazardous  situations. 

•  Stop  timing  when  he  says  the  job  is  finished. 

After  the  Test 

•  Compute  the  score. 

e  Explain  the  scoring  system  to  the  mechanic  and  review  the  results  with  him. 

e  Correct  any  procedural  or  quality  errors,  and  safety  violations  at  this  point. 

•  Have  the  mechanic  sign  the  Test  Grade  Computation  Sheet  agreeing  he  has 
received  feedback. 
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If  60  CM4  -  63N 

TEST  SCORING  NOTES 

Part  1  (Not  timed)  is  scored  by  awarding  1  point  for  each  correct  answer..  ("Correct 
answers"  may  be  given  orally,  or  by  the  mechanic  pointing  to  the  manual 
or  part,  related  to  the  test  question.) 

Part  2  (Timed— 60  minutes  allowed)  is  scored  by  awarding  1  point  each  for: 

-  Maintaining  the  performance  step  sequence* ** 

-  Selecting  the  correct  tool(s) 

-  Using  tool(s)  properly 

-  Working  to  specifications/tolerances 
To  Compute  the  Test  Grade  for  Part  2 

Add  across  each  row  by  page  to  get  score  for  each  performance  component. 

Add  each  column  to  find  mechanic's  total  scores  on  tool  selection  and 
use,  working  to  specs,  etc. 

Add  totals  across  bottom  row,  page  4,  to  get  overall  test  score. 

If  Part  2  is  completed  with  an  overall  score  of  46  within  the  time  blocks 
shown,  add  the  incremental  points  as  shown  in  the  table  below.  If  Part  2  is  completed 
but  outside  the  time  limits,  no  incremental  points  are  awarded. 

Incremental  Points  far  Completion  to  Time  Standard 

If  mechanic  has  a  score  of  at  least  46  on  Part  2 

AND 

-  Has  completed  the  test  in  under  40  minutes-add  33  points  to  his  score 

-  Has  completed  the  test  in  41-48  minutes-add  24  points  to  his  score 

-  Has  completed  the  test  in  49-55  minutes-add  15  points  to  his  score 

-  Has  completed  the  test  in  56-60  minutes-add  6  points  to  his  score 

-  Has  completed  the  test  in  over  60  minutes-add  0  points  to  his  score 


*A  computation  sheet  is  provided  on  page  d. 

**The  TM  will  be  the  reference  authority  for  sequence,  tools,  and  specifications, 
if  appropriate.  If  not,  use  local  standards  and  procedures. 
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M 60  CM4  -  63N 


TEST  GRADE  COMPUTATION  SHEET 


Possible 

Points 

Actual 

Points 

Part  1-10  points 

10 

Part  2 

Done  in  correct  sequence  - 

30 

Selection  of  correct  tool  - 

12 

s 

►  57 

V 

Correct  use  of  tool  - 

12 

Working  to  correct  specs  - 

3^ 

Time  for  Performance 

If  score  for  Part  2  =  46  and  over 

and  Performance  is  under  40  minutes, 
(full  credit) 

add  33  -n 

Performance  in  41-48  minutes  add 
(partial  credit) 

Performance  in  49-55  minutes  add 
(partial  credit) 

24 

15 

>-33 

Select 

only 

one 

v - 

Performance  in  56-60  minutes  add 
(partial  credit) 

6 

Performance  over  60  minutes  add 
(no  credit) 

0-> 

— - 

Total 

Possible  = 
Points 

100 

Actual 
Points : 
Scored 

= 

TESTEE  DATA 

Date _ 

1.  Name  SSN 


2.  _ 

3.  _ 

4.  _ 

5.  _ 

6.  _ 

•  Received  debriefing  X 


Primary  MOS 
AIT 

Number  of  Prior  Task  Experiences 
Number  of  Days  Since  Last  Performance 

Finished 


Time  Taken  for  Test 


Started 
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If  60  CM4  -  63N 


I _ 

PART  1:  GENERAL  BACKGROUND  KNOWLEDGE  RELATED  TO  TM  USE  IN  TEST 

(one  point  for  each  correct  answer) 


Points 


*1.  Show  me  the  TM  that  has  organizational  maintenance  information 
for  the  M60A1  RISE  Passive  2C  engine. 

ANS:  TM  9-2350-257-20  series 

2.  What  is  the  date  of  the  latest  revision? 

ANS:  January  1981 

3.  Which  volume  of  the  TM  contains  the  troubleshooting  procedures 
for  the  2C  engine? 

ANS:  TM  9-2350-257-20-1-1  (Volume  1) 

4.  How  are  the  generators  of  2A  and  2D  engines  cooled? 

ANS:  Air 

5.  How  is  the  generator  of  a  2C  engine  cooled? 

ANS:  Oil 

6.  Where  should  the  oil  inlet  elbow  of  a  2C  generator  be  positioned 
when  you  are  installing  it  on  the  engine? 

ANS:  Down,  at  the  bottom,  or  6  o'clock  position 

7.  Show  me  where  it  lists  the  torque  pressure  for  the  locknut 

of  the  V-band  damp  that  holds  the  generator  tight  to  the  drive 
adapter? 

ANS:  55-65  lb  in  (or  6-7  N  m)  from  TM  9-2350-257-20-1-3 
(Volume  3) 

8.  Show  me  where  it  lists  the  correct  idle  speed  for  a  "ground  hop" 
powerplant  test  run? 

ANS:  700-750  RPM  from  TM  9-2350-257-20-1-2  (Volume  2) 

9.  Will  the  tachometer  operate  during  a  "ground  hop"  powerplant 
test  run? 

ANS:  No  (Not  connected) 

10.  What  gauge  should  show  that  the  generator  is  charging? 

ANS:  Battery  generator 


(Total  Possible  Points  =  10) 

Total  Actual  Points  = 


*Aak  "Have  you  taken  any  previous  NASTQps  for  this  equipment?"  If  YES  -  Award 
2  points  and  go  to  3.  If  NO  -  Start  at  1. 
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Standard 


2a.  (Using  hydraulic  jack), 
position  generator 
and  cradle  assembly 
on  generator  bracket. 

b.  Slide  forward  until 
splined  shaft  engages. 

3.  Ensure  oil  inlet  elbow 
is  at  6  o'clock  position 
and  generator  is  tight 
against  adapter. 

4.  Position  V-band  damp 
around  generator/adap¬ 
ter  mating  flange. 

5a.  Tighten  locknut  on 
V-clamp  with  7/16" 
wrench. 

b.  Tighten  locknut  to 
55-65  lb  in  (6-7  N  m) 
with  7/16"  socket  and 
torque  wrench. 

6.  Align  screw  mounting 
holes  in  cradle,  with 
generator  bracket 
(plastic  hammer  may 
be  used). 

Start  two  screws  (bolts) 
with  flat  washers  in 
holes. 

Tighten  screws  using 
9/16"  socket. 

Tighten  T-bolt  nut 
of  cradle  strap  with 
7/16"  wrench. 

Connect  oil  inlet  hose 
to  elbow. 

Tighten  hose  connection 
using  7/8"  and  3/4" 
wrenches. 


5 


5 


23 


Derived 

from 

TM# 


Performance 
Component  Step 


21a.  Connect  vent  hose  to 
adapter. 


b.  Tighten  connection 
using  9/16"  and  1/2" 
wrenches. 


22a.  Connect  oil  return  hose 
to  adapter. 


b.  Tighten  connection 
using  7/8"  wrench. 


23a.  Connect  oil  return  hose 
to  adapter. 


b.  Tighten  connection 
using  1/2"  and  9/16" 
wrenches. 


24a.  Connect  two  electrical 
connectors. 


b.  Tighten  connector  1 
using  1"  wrench. 

c.  Tighten  connector  2 
using  1-1/8"  wrench. 


25a.  Remove  nut  from  nega¬ 
tive  terminal. 


26.  Install  bus  bar  (and 

attached  ground  straps) 
on  negative  terminal. 


M60  CM4  -  63N 


Checking! 

Done  in  Selection  Correct  Quality 

Correct  of  Cor-  Use  of  Against  Total 
Sequence  rect  Tool  Tool  Standard  Possible  Comments  f 


b.  Clean  terminal  if  needed. 

(1) 


27 .  Install  and  tighten  secur¬ 
ing  nut  with  9/16"  wrench. 


28a.  Remove  nut  from  posi¬ 
tive  terminal. 


b.  Clean  terminal  if  needed. 

(1) 


Possible  Subtotals 


Actual  Subtotals 


not  need  cleaning,  award  2  points. 


-3" 


Derived 

from  Performance 

TM#  Component  Step 


Checking 

Quality 

Against  Total 
Standard  Possible  Com  men' 
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APPENDIX  C 
BRIEFING  SCRIPTS 
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C-l:  BRIEFING  SCRIPT  -  TEST  INTRODUCTION 

Are  you  _  of  _  Company  (or  Bn,  if  necessary)?  I  am 

_ .  I  represent  Anacapa  Sciences,  Inc.  We  are  doing  research  for  the 

Army  to  help  improve  organizational  maintenance  proficiency.  The  company  has 
been  working  at  Fort  Carson  over  the  past  several  years.  We  are  testing  the  repair 
skills  of  a  number  of  mechanics  like  yourself  to  find  better  ways  to  organize 
training.  I  am  required  by  law  to  provide  this  information  about  the  test.  (At  this 
point  cover  the  information  on  the  "Research  Participation  Information  Sheet" 
shown  as  Appendix  C.) 

You  will  be  tested  on  two  tank  repair  tasks,  installing  a  starter  and  installing 
a  generator.  Each  test  has  two  parts.  The  first  part  is  worth  10  points  and  tests 
how  well  you  can  use  the  TM  related  to  the  repair  task.  In  the  second  part  of  the 
test,  worth  90  points,  you  will  actually  do  the  repair.  Two-thirds  of  your  score  will 
be  based  on  accuracy,  doing  the  steps  in  the  right  order  and  using  the  right  tools. 
The  other  one-third  is  based  on  how  soon  you  finish  the  repair  under  the  time  limit. 
You  will  have  one  hour  to  do  each  repair.  You  will  get  a  higher  score  if  you  work 
as  quickly  and  as  accurately  as  you  can.  You  will  install  the  parts  on  the  pack  in 
front  of  you.  Everything  will  be  supplied  including  tools,  TMs,  and  parts.  You  have 
nothing  to  write.  We  will  tell  you  your  scores  after  you've  finished  the  second  test. 
We  want  you  to  do  your  best  so  that  the  test  scores  accurately  represent  your  skill 
level.  Do  you  have  any  questions?  (Answer  his  questions  before  continuing.) 

Before  we  begin,  I  need  to  know: 

•  How  many  times  have  you  replaced  the  starter  on  a  M60  since  completing 
AIT? 

•  How  long  has  it  been  since  you  last  did  this  task?  (Note:  If  the  mechanic 
has  not  done  the  task  since  AIT,  ask  him  how  long  that  has  been  since  he 
last  did  the  repair  at  AIT  or  OJT.) 

•  How  many  times  have  you  replaced  the  generator  on  the  M60A1  tank? 

•  How  long  has  it  been  since  you  last  did  this  task?  (Again,  if  the  mechanic 
has  no  prior  experience  on  this  task,  have  him  indicate  the  time  since  he 
last  replaced  a  generator  during  training.) 
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C-2:  BRIEFING  SCRIPT  -  TM  KNOWLEDGE  TEST  (PART  1) 


Now  we  will  begin  the  starter  (generator)  test.  I  will  first  ask  you  some 
questions  about  the  repair  that  will  require  you  to  use  the  TM.  All  the  questions 
can  be  answered  by  referring  to  the  TM.  You  are  free  to  use  the  TM  as  much  as 
you  like  (now  and  later  in  the  test).  This  part  of  the  test  is  not  timed.  It  is  worth 
10  points.  Do  you  have  any  questions?  (If  not ,  begin  the  test.) 

Since  positioning  the  starter  (generator)  is  a  two-man  job,  Til  act  as  your 
helper  and  will  do  whatever  you  ask  for  that  step.  After  that  you  will  work  on  your 
own. 


C-3:  BRIEFING  SCRIPT  -  HANDS-ON  PERFORMANCE  TEST  (PART  2) 

Now  you  will  take  the  main  part  of  the  test.  It  is  worth  90  points.  Laid  out 
before  you  are  the  tools,  replacement  parts,  and  manuals  that  you  need  to  replace 
the  starter  (generator).  Since  you  will  be  scored  on  the  order  (sequence)  in  which 
you  do  the  steps,  feel  free  to  use  the  TM  to  insure  that  you  are  doing  the  steps  in 
the  right  order. 

Remember,  time  counts,  so  work  quickly  but  as  safely  and  accurately  as 
possible.  If  you  get  stuck  on  a  step,  it  is  better  to  ask  for  help  and  lose  a  few 
accuracy  points  than  to  use  up  all  of  your  time. 

I  will  start  timing  when  you  give  the  signal. 


APPENDIX  D 


SUMMARY  DATA:  MEAN  CRITERION  SCORES  BY 
TEST  AND  JOB  EXPOSURE  CATEGORY 
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STARTER  TEST 


Scoring  Criteria 

JE 

No. 

Overall 

Time 

TM 

Tool 

Tool 

Quality 

Category 

Subjects 

Proficiency 

(Min.) 

Know. 

Sequence 

Select. 

Use 

Check 

0 

16 

78.5 

49.6 

8.8 

26.2 

10.1 

10.3 

9.4 

1 

5 

86.0 

40.4 

9.0 

26.2 

10.5 

10.7 

10.2 

2 

5 

91.4 

41.6 

9.4 

28.4 

10.9 

10.8 

11.2 

3 

5 

93.0 

34.4 

9.6 

28.0 

10.7 

10.7 

11.0 

4 

5 

93.6 

38.6 

10.0 

29.8 

11.0 

11.0 

11.0 

5 

5 

96.6 

33.8 

9.8 

28.4 

11.0 

11.0 

11.4 

6 

5 

97.4 

33.0 

9.8 

29.6 

11.0 

10.9 

11.0 

7 

5 

94.0 

30.8 

9.8 

28.2 

10.8 

10.8 

10.4 

8 

5 

97.6 

30.4 

10.0 

30.2 

11.0 

11.0 

11.8 

9+ 

12 

92.8 

37.0 

9.5 

27.3 

10.8 

10.8 

10.7 

Maximum  Score 

Possible 

100 

10 

31 

11 

11 

12 

GENERATOR  TEST 


Scoring  Criteria 

JE 

No. 

Overall 

Time 

TM 

Tool 

Tool 

Quality 

Category 

Subjects 

Proficiency 

(Min.) 

Know. 

Sequence 

Select. 

Use 

Check 

0 

12 

89.4 

29.7 

8.0 

25.7 

10.3 

11.2 

2.5 

1 

6 

93.8 

27.2 

8.7 

26.8 

11.5 

11.3 

2.5 

2 

5 

97.0 

21.2 

10.0 

27.2 

11.8 

12.0 

3.0 

3 

6 

97.3 

23.2 

9.8 

28.2 

11.7 

11.8 

2.8 

4 

5 

96.6 

19.6 

9.8 

28.0 

11.4 

11.5 

3.0 

5 

5 

99.6 

18.2 

9.8 

30.0 

12.0 

11.9 

3.0 

6 

5 

97.6 

19.2 

9.8 

28.2 

11.7 

11.7 

3.0 

7 

6 

95.3 

20.7 

9.3 

26.5 

11.9 

11.9 

2.5 

8 

7 

96.0 

19.4 

9.7 

27.1 

11.6 

11.8 

2.6 

9+ 

10 

96.5 

21.8 

9.9 

27.9 

11.2 

11.3 

2.6 

Maximum  Score 

Possible 

100 

10 

30 

12 

12 

3 

APPENDIX  E 

SUMMARY  OP  DATA  POOLING  AND  STATBTICAL  ANALYSES 


The  data  were  pooled  in  two  ways  to  simplify  presentation  of  the  findings. 
First,  the  ten-level  experimental  design  was  reduced  to  six  by  combining  subjects 
in  adjacent  pairs  of  JE  categories  between  one  and  eight.  This  pooling  did  not 
change  the  overall  pattern  in  the  data,  but  it  enhanced  the  stability  of  the  means 
by  ensuring  that  at  least  10  subjects  appeared  in  each  category.  Second,  two 
scoring  criteria,  tool  selection  and  tool  use,  were  combined  into  a  single  measure. 
The  obvious  conceptual  link  between  the  two  measures  was  reflected  in  the  data, 
as  subjects’  scores  on  the  two  criteria  were  highly  correlated  (r  =  +.80). 

Where  appropriate,  inferences  regarding  the  relationship  between  JE  and 
repair  skill  were  based  on  statistical  analyses  of  the  data.  To  make  the  text  more 
readable,  the  results  of  the  statistical  tests  are  included  in  this  appendix.  The 
choice  of  tests  and  the  logic  behind  their  use  are  described  below. 

Separate  analyses,  using  the  same  statistical  procedures,  were  made  of  the 
six  maintenance  proficiency  measures  for  each  of  the  two  different  repair  tasks. 
The  statistical  significance  of  the  overall  relationship  between  JE  and  maintenance 
proficiency  was  determined  by  the  F-statistic  from  an  unbalanced  analysis  of 
variance.  The  strength  of  the  relationship  is  indicated  by  the  r  statistic,  the 
proportion  of  variance  in  the  proficiency  measure  explainable  by  JE.1  Next,  the 
magnitude  of  the  increase  in  proficiency  over  the  lower  values  of  JE  was 
statistically  assessed  by  testing  the  significance  of  the  linear  trend  component  in  a 
polynomial  trend  analysis.  Finally,  the  extent  to  which  skill  declined  over  the 
higher  JE  categories  was  inferred  by  examining  the  significance  of  the  quadratic 
trend  component. 

The  results  of  these  analyses  are  given  in  the  following  tables. 


*Hays,  W.  L.  Statistics  for  the  social  sciences.  New  York:  Holt,  Rinehart,  and 
Winston,  1973. 

2Winer,  B.  J.  Statistical  principles  in  experimental  design  (2nd  ed.).  New  York: 
McGraw-Hill,  1971. 


MEASURE:  OVERALL  PROFICIENCY 


Repair  Task 

Analysis 

F-value 

df1 

r2 

p-level 

Starter 

Installation 

Overall  Effect 

11.02 

5/62 

.472 

.0001 

Linear  Trend 

28.62 

1/62 

,523 

.0001 

Quadratic  Trend 

19.81 

1/62 

.36'* 

.0001 

Overall  Effect 

13.57 

5/61 

.532 

.0001 

Generator 

Installation 

Linear  Trend 

27.62 

1/61 

.413 

.0001 

Quadratic  Trend 

29.10 

1/61 

•433 

.0001 

MEASURE: 

TIME  IN  MINUTES 

Repair  Task 

Analysis 

F-value 

df 

r2 

p-level 

Starter 

Installation 

Overall  Effect 

11.39 

5/62 

.48 

.0001 

Linear  Trend 

32.18 

1/62 

.56 

.0001 

Quadratic  Trend 

18.23 

1/62 

.32 

.0001 

Overall  Effect 

6.21 

5/61 

.34 

.0001 

Generator 

Installation 

Linear  Trend 

17.05 

1/61 

.55 

.0001 

Quadratic  Trend 

11.99 

1/61 

.39 

.0001 

MEASURE: 

TOTAL  ACCURACY 

Repair  Task 

Analysis 

F-value 

df 

r2 

p-level 

Starter 

Installation 

Overall  Effect 

5.87 

5/62 

.32 

.0002 

Linear  Trend 

10.11 

1/62 

.34 

.0023 

Quadratic  Trend 

16.30 

1/62 

.56 

.0002 

Overall  Effect 

7.27 

5/61 

.37 

.0001 

Generator 

Installation 

Linear  Trend 

12.08 

1/61 

.33 

.0009 

Quadratic  Trend 

18.41 

1/61 

.51 

.0001 

^Degrees  of  freedom  for  numerator,  denominator,  respectively. 
Proportion  of  total  variance  explained  by  the  effect. 

3 

Proportion  of  effect  variance  explained  by  the  trend  component. 


MEASURE:  TM  KNOWLEDGE 


Repair  Task 

Analysis 

F-value 

df 

r2 

p-level 

Starter 

Installation 

Overall  Effect 

3.36 

5/62 

.21 

.0095 

Linear  Trend 

7.88 

1/62 

.47 

.0067 

Quadratic  Trend 

7.21 

1/62 

.43 

.0093 

Overall  Effect 

10.86 

5/61 

.47 

.0001 

Generator 

Installation 

Linear  Trend 

30.57 

1/61 

.56 

.0001 

Quadratic  Trend 

14.11 

1/61 

.26 

.0004 

MEASURE:  FOLLOWING  SEQUENCE 

Repair  Task 

Analysis 

F-value 

df 

r2 

p-level 

Starter 

Installation 

Overall  Effect 

2.59 

5/62 

.17 

.0338 

Linear  Trend 

3.29 

1/62 

.25 

.0746 

Quadratic  Trend 

8.33 

1/62 

.64 

.0054 

Overall  Effect 

2.98 

5/61 

.20 

.0181 

Generator 

Installation 

Linear  Trend 

4.15 

1/61 

.28 

.0460 

Quadratic  Trend 

5.31 

1/61 

.36 

.0246 

MEASURE: 

TOOL  SELECTION/USE 

Repair  Task 

Analysis 

F-value 

df 

r2 

p-level 

Starter 

Installation 

Overall  Effect 

3.81 

5/62 

.23 

.0046 

Linear  Trend 

7.47 

1/62 

.39 

.0082 

Quadratic  Trend 

8.42 

1/62 

.44 

.0051 

Overall  Effect 

4.57 

5/61 

.27 

.0014 

Generator 

Installation 

Linear  Trend 

5.49 

1/61 

.24 

.0224 

Quadratic  Trend 

13.02 

1/61 

.57 

.0006 

MEASURE:  QUALITY  CHECK 


Repair  Task 

Analysis 

F-value 

df 

r2 

p-level 

Overall  Effect 

3.48 

5/62 

.22 

.0078 

Starter 

Installation 

Linear  Trend 

6.00 

1/62 

.34 

.0171 

Quadratic  Trend 

8.41 

1/62 

.48 

.0052 

Generator 

Installation 

Overall  Effect* 

0.95 

5/61 

.07 

.4581 

^Nonsignificant  F-value  invalidates  further  statistical  testing  of  the  trend  components. 


